The present invention relates generally to imaging and vision systems, more particularly to foveal imaging systems, and, even more particularly, to a CMOS foveal image sensor integrated circuit. The present invention also provides a novel photo charge normalization technique which enables use of the same charge amplifier with different size pixels while simultaneously providing a wide dynamic response to the incoming light.
In contrast to the uniform acuity of conventional imaging systems, virtually all advanced biological vision systems sample the scene in a space-variant fashion. Retinal acuity varies by several orders of magnitude within the field-of-view (FOV). The region of the retina with notably high acuity, called the fovea, is typically a small percentage of the overall FOV (about 5%), centered at the optical axis. The wide FOV, with lower peripheral acuity, and high acuity fovea results in a much smaller data set than supporting the entire FOV uniformly at high acuity. Inherent with space variant sampling is the context-sensitive articulation of the sensor""s optical axis whereby the fovea is aligned with relevant features in the scene. These features can be targets such as predators, preys or classification features on the targets themselves. Space-variant sampling and intelligent gaze control together with multi-resolution image analysis are collectively called foveal vision.
The key benefits of foveal vision are its simultaneously achieving wide FOV, high resolution and fast frame rates. This makes it particularly well suited for time critical (real-time) applications traditionally associated with high bandwidth uniform vision, such as those of fast automata and active pursuit scenarios.
In general, foveal vision offers a visual information acquisition power that is superior to uniform acuity vision. A metric of visual information acquisition power is the product of FOV and spatial resolution and frame rate. In a foveal system, the metric is computed as the product of the total (i.e. peripheral) FOV, the spatial resolution at the fovea, and the overall frame rate.
The higher central resolution of foveal vision improves target classification by increasing recognition confidence and reducing classification error. The wider FOV of foveal vision reduces search and detection time, and improves the reliability of target tracking and pursuit.
The faster frame rate improves the behavioral response of vision driven autonomous agents, i.e. improves the system""s ability to detect brief events and fast phenomena without temporal aliasing effects. Allowing shorter time between images improves the temporal (frame-to-frame) correlation of image features for more robust motion perception, and permits the use of simpler kinematics prediction models, which need not predict as far into the future. These benefits support more accurate navigation, target tracking and intercept, obstacle avoidance, and smoother control of articulations (better hand-eye coordination).
The theoretical basis for the general concepts of a foveal machine vision system was set forth in a doctoral thesis entitled Foveal Machine Vision Systems, by Cesar Bandera, submitted to the Department of Electrical and Computer Engineering, State University of New York at Buffalo in August, 1990. Although this thesis presented the theoretical underpinnings for a foveal machine vision system, and also suggested the need for a VLSI implementation of the invention, heretofore no one has solved the technical problems associated with such an integrated circuit reduction to practice.
Further background information related to the present invention can be found in the following references:
[1] E. R Fossum. xe2x80x9cActive pixel sensors-are CCD""s dinosaurs?xe2x80x9d in Charge-Coupled Devices and Solid-State OpticalSensors III Proc.SPIE. vol. 1900, February 1993, pp.2-14
[2] R. Nixon, S. Kemeny, C. Staller, and E. Fossum, xe2x80x9c128xc3x97128 CMOS photodiode-type active pixel sensor with on-chip timing, control and signal chain electronicsxe2x80x9d in Charge-Coupled Devices and Solid-State OpticalSensors V. Proc.SPIE, vol. 2415, February 1995, paper no. 34
[3] S. Mendis, S. Kenemy, R. Gee, B. Pain, Q. Kim, and E. Fossum, xe2x80x9cProgress in CMOS active pixel image sensors,xe2x80x9d in Proc.SPIE, vol. 2172, February 1994, pp. 19-29
[4] S. Mendis, S. Kenemy, R. Gee, B. Pain, C. Staller, Q. Kim, and E. Fossum, xe2x80x9cCMOS active pixel image sensor for highly integrated imaging systemsxe2x80x9d, in IEEE Journal of Solid-State Circuits, February 1997, pp. 187-197
[5] C. Bandera and P. Scott, xe2x80x9cFoveal machine vision systems.xe2x80x9d, IEEE International Conference on Systems, Man, and Cybernetics. Cambridge, Mass., November 1989, pp.569-599
[6] E. Franchi, M. Tartagni, R. Guerrieri, and G. Baccarani, xe2x80x9cRandom access analog memory for early visionxe2x80x9d, in IEEE Journal of Solid-State Circuits, vol. 27, no.7, July 1992
[7] T. Ebben, CCD electronics, Ball Aerospace Systems Group. 1996
[8] D. Wobschall, Circuit design for electronic instrumentation, 2nd edition, McGraw-Hill
The present invention comprises a foveal image sensor integrated circuit, having a first plurality of CMOS active pixels of a first size arranged in a central fovea region on a substrate of the circuit, a second plurality of CMOS active pixels of a second size arranged in a first peripheral ring about the central fovea region; and, a normalization circuit operatively arranged to normalize a photocharge applied to each of the different size pixels such that an output signal of each of the second size pixels is equal to an output signal of each of the first size pixels when the first and second size pixels are subjected to equal illumination.
A general object of the present invention is to provide a CMOS integrated circuit foveal image sensor.
A secondary object of the present invention is to provide a charge normalization scheme and circuit for normalizing the output signals of pixels of varying sizes in a foveal sensor array.
These and other objects, features and advantages of the present invention will become readily apparent to those having ordinary skill in the art upon a reading of the following detailed description of the preferred embodiment in view of the attached drawings and appended claims.